\  19951012  045 


AN  INFORMATION  THEORY  APPROACH  TO  THREE  INCOHERENT 
INFORMATION  PROCESSING  SYSTEMS 


Edward  R,  Dowski,  Jr, 

Imaging  Systems  Laboratory 
Department  of  Electrical  &  Computer  Engineering 
University  of  Colorado 
Boulder,  Colorado 
80309-0425 


1.  INTRODUCTION 

Many  incoherent  optical/digital  systems  can  be  used 
for  non-imaging  purposes,  such  as  passive  ranging. 
These  systems  cannot  effectively  be  analyzed  or  de¬ 
signed  in  terms  of  traditional  image-forming  sys¬ 
tems.  Instead,  such  systems  should  be  analyzed  in 
terms  of  information  theory.  Through  mathemat¬ 
ical  modelling  of  the  S2tmpled  image,  information 
theory  can  be  used  to  optimize  a  given  system. 

The  Fisher  Information  matrix  and  the  Cramer- 
Rao  bound  are  two  widespread  and  tractable  mea¬ 
sures  of  the  information  content  of  a  signal.  By 
analyzing  information  processing  systems  in  terms 
of  these  measures  of  information,  illuminating  con¬ 
clusions  that  are  related  to  optimum  design  can  be 
found.  Such  conclusions  lead  to  necessary  condi¬ 
tions  that  particular  types  of  information  process¬ 
ing  systems  must  possess.  We  consider  these  neces¬ 
sary  conditions  for  three  incoherent  optical  systems, 
namely  passive  range  estimation,  extended  depth  of 
field,  and  passive  range  detection. 

2.  FISHER  INFORMATION  AND  THE 
CRAMER-RAO  BOUND 

Let  the  parameters  desired  from  a  given  estimation 
system  be  denoted  by  the  length  p  vector  ^ 

(1) 

where  the  noiseless  measurement  is  some  vector  func¬ 
tion  of  these  parameters,  say  The  actual 

measurement  in  any  real  system  will  always  be  cor¬ 
rupted  by  noise.  The  limit  of  this  noise  will  be  signal 
dependent  shot  noise  or  detector  quantization  noise. 
We  assume  a  zero  mean  white  gaussian  noise  with 
variance  cr^.  Our  ability,  on  the  average,  to  estimate 
^  is  bounded  by  the  Cramer-Rao  bound  [1,  2,  3], 


This  bound  can  describe  both  biased  and  unbiased 
estimators.  We  consider  here  only  unbiased  estima¬ 
tors.  The  variance  of  any  unbiased  estimator  of  one 
component  of  say  is  bounded  below  as 

var{^i)  >  (2) 


where  J(^)  is  the  Fisher  Information  matrix  of  the 
parameter  ^  [2,  3].  Let  be  the  probability 

density  function  for  the  observed  noisy  data  y.  The 
Fisher  information  matrix  is  then  given  by 


J{±)  =  E  In  p(y;^)  In  p(y;$)  (3) 


where  E  denotes  expected  value.  Under  the  zero 
mean  white  gaussian  noise  assumption  (3)  reduces 
[4]  to 


0’-  =  ^  (4) 


The  matrix  G  is  called  a  sensitivity  matrix.  Assume 
that  the  parameter  ^  is  partitioned  into  two  sets  so 
that  ^  partitioning  the  matrix  G  of 

(4)  as 


■G-[GiG2];  Gi 


~W’ 


Go 


dxi±) 


(5) 


we  can  show  that  the  inverse  of  the  Fisher  Informa¬ 
tion  matrix  of  (3)  is  given  by  [4] 

(6) 

where 

P^,  =  I-Gi{GjGi)-^Gj 

is  a  projection  matrix  projecting  onto  the  space  or¬ 
thogonal  to  the  space  spanned  by  the  matrix  Gi ,  or 


[Gf  Gi] 


-1 
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<  Gi  >.  The  identity  matrix  is  given  by  7.  Ap¬ 
plication  of  (6)  to  specific  information  processing 
systems  leads  to  estimates  of  the  theoretical  perfor¬ 
mance  limit  for  specific  systems,  while  also  deter¬ 
mining  necessary  the  conditions  for  such  systems. 

3,  SINGLE-IMAGE,  SINGLE-LENS, 
PASSIVE  RANGE  ESTIMATION 
SYSTEMS 

Incoherent  Single-image,  single-lens,  passive  range 
estimation  systems  code  the  object  range  informa¬ 
tion  into  unambiguous  spatial  information  at  the 
image.  Such  systems  should  be  insensitive  to  the 
characteristics  of  the  particular  object.  The  noise¬ 
less  sampled  image  model  of  this  system  is 

X  =  h(^)  ♦  «  =  F{xl))u  (7) 

where  the  system  point  spread  function  (PSF)  or 
impulse  response  is  given  by  h(V’).  This  PSF  is 
characterized  by  the  misfocus  or  normalized  range 
parameter  xj).  The  symbol  ♦  denotes  convolution. 
The  matrix  F(V’)  is  a  convolution  matrix  contain- 
ing  h(^)  as  elements.  The  unknown  object  is  given 
by  «. 

The  unknown  parameters  of  the  noiseless  sam¬ 
pled  data  are  the  normalized  range  rj)  and  the  object 
u.  The  system,  described  by  h(‘0)  and  is  as¬ 

sumed  to  be  known.  The  unknown  parameters  can 
be  grouped  as 

^  =  ±2-- 


Since  the  matrix  F(^)  can  be  approximated  by 
a  circulant  matrix,  the  eigenvalues  of  F(^)  are  ap¬ 
proximately  the  DFT  values  of  h(^)  [5].  The  de¬ 
nominator  of  (9)  can  then  be  approximated  as 

(10) 

where 


otherwise 


(11) 


and  where  77(‘0,  and  are  the  DFTs  of 

the  sampled  vectors  h(rp)  and  u  respectively.  No¬ 
tice  that  is  the  optical  transfer  function 

(OTF)  of  the  sampled  system  with  misfocus  while 
U{e^^)  is  the  spatial  frequency  spectrum  of  the  sam¬ 
pled  object. 

From  (9)  and  (10),  a  necessary  condition  for  a 
passive  range  estimation  systems  is  that  the  OTF 
must  contain  zeros  that  are  a  function  of  misfocus 
or  normalized  range  It  is  impossible  to  build  a 
general  passive  range  estimation  system  whose  OTF 
does  not  contain  zeros  as  a  function  of  misfocus. 

With  a  priori  information  of  the  object  spatial 
frequency  spectrum,  from  (10)  a  desired  condition 
on  the  system  is  that  the  magnitude  squared  varia¬ 
tion  of  the  OTF  should  be  a  matched  filter  for  the 
expected  object  spatial  frequency  power  spectrum. 


4.  EXTENDED  DEPTH  OF  FIELD 
SYSTEMS 


With  this  partitioning  of  the  parameters,  the  sensi¬ 
tivity  matrices  from  (5)  are  found  to  be 

Gi  =  G2  =  F{^)  (8) 

From  (6)  the  Cramer- Rao  bound  on  estimating  the 
normalized  range  tp  with  unknown  u  is  given  by 


In  an  incoherent  extended  depth  of  field  system,  un¬ 
known  objects  are  estimated  without  knowledge  of 
misfocus  or  object  range.  The  model  for  the  noise¬ 
less  sampled  image  is  identical  to  the  model  of  the 
passive  ranging  system 

X  =  h(^)  ♦  u  =  F{'ip)u  (12) 


-  cr  4) 


(9) 


where  is  a  projection  matrix  projecting  onto 

the  subspace  orthogonal  to  <  F{'4))  >.  From  (9) 
we  conclude  that  for  passive  range  estimation  to 
be  possible  must  not  be  a  rank-zero  projec¬ 

tion,  or  equivalently  F{^)  must  not  be  full  rank. 
The  Cramer-Rao  bound  of  (9)  also  shows  that  the 
variation  of  F(^)  with  ‘tp,  in  the  intersection  of  the 
rank-one  subspace  <  u>  and  the  subspace  orthog¬ 
onal  to  <  F(^)  >,  must  be  large  for  accurate  range 
estimation. 


The  difference  between  the  passive  ranging  prob¬ 
lem  and  the  extended  depth  of  field  problem  is  that- 
instead  of  estimating  object  range  'ip  without  knowl¬ 
edge  of  the  object  «,  the  unknown  object  u  is  esti¬ 
mated  without  knowledge  of  the  range  ip. 

Partition  the  parameter  vector  as  in  the  passive 
ranging  problem.  The  sensitivity  matrices  Gi  and 
G2  are  then  the  same  as  in  (8).  The  Cramer-Rao*. 
bound  on  the  estimate  of  the  unknown  object  u  from 
(6)  is  then 


var{^  > 


trace(f  (^)T  F(i>)) 


□ 


(13)  es 


i 
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where  Pq^  is  the  projection  matrix  projecting  onto 
the  subspace  orthogonal  to  <  Gi  >.  The  relation¬ 
ship  of  (13)  then  implies  that  the  variation  of  F{il>) 
with  in  the  rank-one  subspace  <  u  >,  should  be 
zero.  Additionally,  the  eigenvalues  of 
should  be  large. 

The  denominator  of  (13)  can  be  placed  in  terms 
of  the  OTF  and  spatial  frequency  spectrum  of  w,  as 
in  (10),  by 


trace(J’(V-r  ^  j P{^<‘)\H{r}>,e?^)\^d9 

(14) 

where 


if|^P'(V-,e^'’)P(e>®)|  =  0 

otherwise 


(15) 

From  (13)  and  (14),  the  necessary  condition  on  a 
reliable  extended  depth  of  field  system  is  that  the 
variation  of  the  OTF  with  misfocus  should  be  zero 
over  the  spatial  frequencies  where  the  expected  ob¬ 
ject  has  non-zero  power.  A  desired  condition  is  that 
the  power  of  the  OTF  should  be  maximized  where 
its  variation  with  misfocus,  or  the  object  spatial 
frequency  spectrum,  is  zero.  From  (15),  the  per¬ 
formance  of  this  type  of  system  is  independent  of 
the  unknown  object  if  the  object  spatial  frequency 
spectrum  contains  no  zero  values  in  the  passband  of 
the  system. 


5.  PASSIVE  RANGE  DETECTION 
SYSTEMS 

In  an  incoherent  passive  range  detection  system  the 
sum  of  images  of  unknown  objects  from  a  number 
of  known  ranges  is  sampled.  This  can  be  modelled 
as 

X  =  F(ipi)ui  (16) 

t  i 

The  normalized  range  values  'tpi  are  assumed  known 
while  the  objects  at  the  different  ranges,  are  as¬ 
sumed  unknown.  For  simplicity  consider  the  imag¬ 
ing  at  two  ranges,  i  =  1,2.  The  results  can  be 
generalized  to  any  number  of  ranges. 

Partition  the  parameter  vector  for  the  sampled 
image  as 

=  ^2  =  «2  (17) 

The  sensitivity  matrices  of  (5)  then  become 

Gi  =  F(^i),  G2  =  F(^2)  (18) 


The  Cramer-Rao  bound  on  the  estimation  of 
with  unknown  is  then 


2 

a  F(*)) 

The  projection  matrix  of  (19)  projects  onto  the  sub¬ 
spaces  orthogonal  to  <  ^^(^2)  >•  For  reliable  op¬ 
eration  at  distinct  ranges  and  i/?2)  Iho  system 
should  possess  orthogonal  subspaces  <  >  and 

<  F{rp2)  >♦  And,  the  eigenvalues  of  F(V’i)^F(^i), 
within  the  subspace  <  >?  should  be  maxi¬ 

mized.  Parallel  statements  to  those  above  can  be 
made  for  the  estimation  of  object  U2^ 

Again,  this  bound  can  be  placed  in  terms  of  the 
OTF  and  spatial  frequency  spectrum  of  the  object 
as 


trace(F(^i)’’  Pf(v-3)  ■^(V’l))  « 

j  (20) 


where 


^  “  1  0  otherwise 


From  (19)  and  (20),  the  necessary  condition  for  pas¬ 
sive  range  detection  systems  is  that  the  expected 
OTFs,  as  a  function  of  misfocus,  should  form  an  or¬ 
thogonal  set.  A  desired  condition  is  that  the  total 
power  of  each  expected  OTF  should  be  maximized 
and  equalized  among  all  expected  OTFs.  Notice 
that  the  performance  of  this  type  of  system  is  inde¬ 
pendent  of  the  unknown  objects. 
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